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Systems  and  Software  Engineering 


An  Organizational  Construct 


Acquisition  program  excellence  through  sound  systems  and  software  engineering 
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DoD  Engineering 
Center  of  Excellence 

•  Support  Acquisition  Success 

•  Improve  State-of-the-Practice 
of  Engineering 

•  Leadership,  Outreach  and 
Advocacy 

•  Foster  Resources  to  Meet 
DoD  Needs 
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•  Support  Acquisition  Success 

-  Ensure  effective  and  efficient  software  solutions  across  the  acquisition 
spectrum  of  systems,  SoS  and  capability  portfolios 

•  Improve  the  State-of-the-Practice  of  Software  Engineering 

-  Advocate  and  lead  software  initiatives  to  improve  the  state-of-the- 
practices  through  transition  of  tools,  techniques,  etc. 

•  Leadership,  Outreach  and  Advocacy 

-  Implement  at  Department  and  National  levels,  a  strategic  plan  for 
meeting  Defense  software  requirements 

•  Foster  Software  Resources  to  meet  DoD  needs 

-  Enable  the  US  and  global  capability  to  meet  Department  software 
needs,  in  an  assured  and  responsive  manner 


Promote  World-Class  Leadership  for  Defense  Software  Engineering 
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Getting  Started  —  What  are  we  Doing? 


•  Identifing  software  issues,  needs 

-  Software  Industrial  Base  Study 

-  NDIA  Top  Software  Issues  Workshop 

-  Defense  Software  Strategy  Summit 

•  Creating  opportunities,  partnerships 

-  Established  network  of  Government  software  POCs 

-  Chartered  the  NDIA  Software  Committee  and  Expert  Panel 

-  Information  exchanges  with  Government,  Academia,  and  Industry 

•  Executing  focused  initiatives 

-  Providing  software  support  to  acquisition  programs 

-  Foundational  assessment  of  software  policy/guidance 

-  Study  of  Software/SE  integration 

-  Software  Engineering  reference  curriculum 

-  Engineering  for  System  Assurance 

-  SoS  Systems  Engineering  Guide 

-  CMMI  Integrity,  CMMI-ACQ,  CMMI  Guidebook 
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Top  Software  Issues * 


1.  The  impact  of  requirements  upon  software  is  not  consistently  quantified  and 
managed  in  development  or  sustainment. 

2.  Fundamental  system  engineering  decisions  are  made  without  full  participation 
of  software  engineering. 

3.  Software  life-cycle  planning  and  management  by  acquirers  and  suppliers  is 
ineffective. 

4.  The  quantity  and  quality  of  software  engineering  expertise  is  insufficient  to 
meet  the  demands  of  government  and  tne  defense  industry. 

5.  Traditional  software  verification  techniques  are  costly  and  ineffective  for 
dealing  with  the  scale  and  complexity  of  modern  systems. 

6.  There  is  a  failure  to  assure  correct,  predictable,  safe,  secure  execution  of 
complex  software  in  distributed  environments. 

7.  Inadequate  attention  is  given  to  total  lifecycle  issues  for  COTS/NDI  impacts 
on  lifecycle  cost  and  risk. 


*NDIA  Top  Software  Issues  Workshop 
August  2006 
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DoD  Software  —  What  We're  Seeing* 


•  Software  systemic  issues  are  significant  contributors  to  poor  program 
execution 

-  Software  requirements  not  well  defined,  traceable,  testable 

-  Immature  architectures,  COTS  integration,  interoperability,  obsolescence 
(electronics/hardware  refresh) 

-  Software  development  processes  not  institutionalized,  planning  documents 
missing  or  incomplete,  reuse  strategies  inconsistent 

-  Software  test/evaluation  lacking  rigor  and  breadth 

-  Schedule  realism  (compressed,  overlapping) 

-  Lessons  learned  not  incorporated  into  successive  builds 

-  Software  risks/metrics  not  well  defined,  managed 


*  Based  on  ~65  program  reviews  to  date 
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SW  Issue/ GAP  Workshop  Findings 


Ongoing  SW  Initiatives  (w/owners)  and  Gaps  binned  to  Focus  Groups 


SYS 


Next  Steps 


Near  Term: 

•  Determine  metrics  for  each  of  the  6  Focus  Areas 

-  Based  upon  source  reports  (ie.  SW  Summit,  Top  Issues,  PSRs,  Historical  SW  Studies) 

•  Coordinate  ongoing  initiatives  (via  Working  Group  Participation,  Defense 
Software  in  Acquisition  Collaborators) 

-  Support  and/or  leverage  initiatives  where  appropriate 

-  Provide  visibility  across  the  Department 

•  Determine  action  plans  for  each  gap  considering: 

-  Priority 

-  Near  Term/Long  Term  impacts 

-  NDIA  SW  Committee,  others,  interest  in  accepting  gap(s) 

•  Engage  other  communities  and  participants 

-  IT,  Business,  Research 

Over  Time: 

•  Reassess  ongoing  initiatives  against  focus  area  metrics 

-  Determine  new  gaps,  or  additional  effort  required  to  address  core  issues 

•  Reassess  focus  area  metrics  against  systemic  software  issues 

-  From  future  SW  Summits,  Systemic  Analysis,  etc... 
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Software  In  Acquisition  Workshop 
October  16-17,  2007 

•  Purpose:  Off-year  workshop  (Summit  held  every  2yrs) 

-  Measure  progress  on  initiatives  against  known  issues 

-  Collaborate  on  gaps 

-  Identify  emerging  issues,  or  adjust  measures  for  existing  ones 

•  Format: 

-  Leadership  updates 

-  Panel  on  hot  topics  (perhaps  embedded  vs  IT  software*) 

-  Presentations  to  share  progress,  experiences 

-  General  discussion/summary  for  each  focus  group 

•  Audience:  DoD  programs,  practitioners,  industry, 

FFRDC,  developers/integrators,  academia 

-  Community  forum  focused  on  software  in  acquisition 

•  Location:  Washington,  DC  area,  TBD 

*based  upon  ideas  generated  at  SSTC 
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OUSD(A  T&L )/SSA 
FOCUSED  INITIA  TIVES 

Opportunities  for  Collaboration! 
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System  Assurance  Context  for  the  PM 


System  Assurance  Definition 

Level  of  confidence  that  a  system  functions  as  intended,  is  free  of 
exploitable  vulnerabilities,  and  protects  critical  program  information 
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Path  Forward 


STRENGTH  THROUGH  INDUSTRY  &  TECHNOLOGY 


•  Create  a  'framework'  to  integrate  multiple  security  disciplines  and  policies 

-  Leverage  5200.39:  expand  CPI  definition  to  include  system  assurance 
and  total  life  cycle 

•  Use  the  Program  Protection  Plan  (PPP)  to  identify  CPI  and  address  assurance 
for  the  program 

-  Link  plans  (e.g.,  Anti-Tamper,  Software  Protection,  System  Engineering, 
Assurance  Case) 

•  Modify  Acquisition  and  System  Engineering  guidance  to  integrate  system 
assurance  across  the  lifecycle 

-  Milestone  Decision  Authority  visibility 

-  Guidebook  on  Engineering  for  Assurance  for  program 
managers/engineers 


Raise  the  bar: 

Awareness 

-  Knowledge  of  the  supply  chain 

-  Who  has  access  to  our  critical  assets 

Protection 

-  Protect  critical  assets  through  security  practices 

-  Engineer  our  systems  for  assurance 
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System  Assurance  Guidebook 
Project  Description 


STRENGTH  THROUGH  INDUSTRY  &  TECHNOLOGY 


Issue:  Systems  are  vulnerable  to  malicious  tampering 


•  Project  Description: 

-  Provide  practical  guidance  on  augmenting  systems  engineering  practice  for 
system  assurance 

-  Synthesize  existing  knowledge  from  organizations,  standards  and  best 
practices 

-  Recap  concepts  from  standards 

•  Opportunity  for: 

-  Practitioners,  academe  who  implement  systems  engineering,  assurance, 
safety,  security,  program  protection,  etc.  into  processes  and  programs 

•  The  project  addresses 

-  Integration  of  assurance  guidance  and  practices  into  systems  engineering 

•  Product: 

-  Guidebook  on  Engineering  for  System  Assurance 

•  Outcome  Goal: 

-  Intent  is  to  yield  assured  program  /  system  with  demonstrable  evidence  of 
assurance 
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System  of  Systems 
Project  Description 


Issue:  No  common  definition,  or  guidance  for  SoS 

•  Project  Description: 

-  Effort  led  by  the  Office  of  the  Secretary  of  Defense 

-  Collaborative  Approach  with  DoD,  Industry,  Academia 

•  Purpose 

-  6  month  effort  addressing  areas  of  agreement  across  the  community 

-  Focus  on  technical  aspects  of  SE  applicable  across  SoS  management  constructs 

-  Vehicle  to  capture  and  debate  current  SoS  experience 

•  Audience 

-  Program  Managers  and  Lead/Chief  Engineers 

•  The  project  addresses 

-  Considerations  for  engineering  above  a  system  level 

•  Product: 

-  SoS  Engineering  Guide,  vl.O,  Fall  2007 

•  Outcome  Goal: 

-  Program  managers/chief  engineers  have  requisite  knowledge  to  manage  SoS 
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System  of  Systems 


•  Why  SoS 

-  Changing  operations  -  changing  threats  and  concepts  mean  that  new  (ad 
hoc)  SoS  configurations  will  be  needed  to  address  changing,  unpredictable 
operational  demands 

-  Legacy  -  given  defense  budget  projections,  current  systems  will  be  part  of 
the  defense  inventory  for  the  long-term  and  need  to  be  factored  into  any 
approach  to  SoS 

•  Observations/challenges 

-  Scale  -  size  of  defense  enterprise  makes  a  single  integrated  architecture 
infeasible 

-  Ownership/Management  -  individual  systems  are  owned  by  the  military 
component  or  agencies,  introducing  constraints  on  management  and  SE 

-  Criticality  of  software  -  SoS  typically  focus  on  integration  across  systems 
through  cooperative  or  distributed  software 

-  Role  of  network  -  conceptually  DoD  SoS  will  be  network-based;  budgetary 
and  legacy  challenges  could  lead  to  uneven  implementation 
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System  of  Systems 
The  Management  Challenge 


SoS: 

Within 
Single 
Organization 


Joint  SoS: 

Interdependencies 

Across 

Multiple 

Organizations 


Political  and  Cost  Considerations  impact  on 

Technical  Issues 


SYSTE 


SW  Engineering  Graduate  Curriculum 

Project  Description 


Issue:  There  is  no  commonly  accepted  structure  or 
content  for  graduate  software  engineering  education 


•  Project  Description: 

-  Develop  a  core  curriculum  and  core  competencies  for  software  engineering 

•  Opportunity: 

-  Industrial  and  government  workforce  customers  of  SWE  graduate  education 

-  Academics  who  provide  SWE  and  SE  graduate  education 

-  Professional  societies  with  a  vested  interest  in  SWE  and  SE  graduate  education 

•  The  project  addresses 

-  Inconsistencies  in  software  graduate  degrees 

-  Poor  definition  of  labor  categories  and  software  expertise 

-  The  divide  between  systems  and  software  engineers  in  industry,  government,  and 
academia 

-  The  project  will  integrate  SE  principles  and  practices  into  a  SWE  curriculum. 

•  Product: 

-  An  approved  curric  that  can  be  adopted  by  the  community  (industry,  academia, 
associations) 

•  Outcome  Goal(s): 

-  Software  engineers  have  a  more  consistent  training  base 
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SE/SW  Process  Integration 
Project  Description 


|C|S|S|E| 


Issue:  SE  and  SW  have  not  been  well  integrated  on  projects 


•  Project  Description: 

-  Study  SE  and  SW  processes,  capture  ongoing  harmonization  efforts 

-  Assess  current  guidance 

-  Identify  opportunities  for  better  integration 

•  Opportunity  for: 

-  SE  and  SW  process  owners  or  practitioners 

-  Academe  who  teach/study  SE  and  SW 

•  The  project  addresses 

-  Integration  of  SW  with  requirements,  risk  management  and  other  SE 
technical  and  management  processes 

•  Product: 

-  Report  and  recommendations  for  SW  policy,  guidance,  and  tools  to 
better  integrate  with  SE  and  Acquisition 

•  Outcome  Goal: 

-  Software  is  a  major  factor  in  engineering  design  and  acquisition 
management  decisions 
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CMMI -Acquisition 
Project  Description 


Issue:  Acquirers  lack  an  appraisable  model  for  acquisition  PI 


•  Project  Description: 

-  Using  GM  CMMI  for  Outsourcing;  pilot  and  generate  CMMI-ACQ 

-  Involve  broad  set  of  acquisition  stakeholders  to  ensure  wide  application 

•  Opportunity  for: 

-  Process  improvement  stakeholders 

-  Acquiring  organizations 

•  The  project  addresses 

-  Identification  of  key  acquirer  activities  and  products 

-  Amplification  of  CMMI  core  practices  to  capture  acquirer  considerations 

•  Product: 

-  CMMI  model  for  Acquisition  (built  on  CMMI  foundation  for  consistency 
with  CMMI-DEV) 

•  Outcome  Goal: 

-  Acquisition  organizations  implement  best  practices,  and  institute 
organizational  process  improvement 
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Additional  areas  for 
collaboration... and  attention 

•  Additional  projects 

-  Software  earned  value  guidance 

-  Software  metrics 

-  Software  knowledge  portal 

•  Some  key  gaps  remaining 

-  Software  sustainment 

-  Software  test 

-  Software  estimation 

-  Software  risk  identification 
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Our  Challenge 


•  Given  the  shortage  of  software  resources  and  critical  software 
reliance 

-  We  cannot  afford  to  be  stovepiped 

-  We  must  integrate  across  cross-functional  perspectives  to  improve  our 
software  capability 

•  We  must  focus  on  long  standing  software  issues 

-  Leverage  ongoing  activities  to  make  a  difference 

-  Invest  in  collaborative  efforts  where  there  are  gaps 

•  Now... 

-  Work  together  to  address  software  issues 

-  Submit  papers  for  the  October  Conference 

-  Join  the  DoD  SIA  Collaborators 

-  Contribute  to  ongoing  initiatives:  SW  Curriculum,  SoS,  Sys  Assurance, 
CMMI  Guides,  more 

-  Contact  us  at  ATL-SSA@osd.mil 

Become  a  DoD  Center  of  Excellence 
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